Change ChainDescriptions into blobs #3787

bart-linera · 2025-04-16T20:00:10Z

Motivation

This will allow for easier initialization of the network at genesis, among other things.

Proposal

ChainDescription will become a blob, and OpenChain messages will be removed. Calling open_chain will create the blob, and the chain will actually get initialized when a block is proposed for it.

Test Plan

Should be covered by CI.

Release Plan

Nothing to do / These changes follow the usual release cycle.

Links

ma2bd · 2025-04-16T20:04:15Z

linera-base/src/data_types.rs

+    /// The epoch in which the chain is created.
+    pub epoch: Epoch,
+    /// Blob IDs of the committees corresponding to epochs.
+    pub committees: BTreeMap<Epoch, Vec<u8>>,


Why Vec<u8> instead of a blob id?

Then this blob depends on other blobs, which complicates chain initialization, so we thought we'd put the full committees in here for now. We could change that as an optimization later.

The comment is inaccurate then, right?

Ah, right! Yes, it is.

ma2bd · 2025-04-16T20:05:16Z

linera-base/src/data_types.rs

+    }
+}
+
+/// The configuration for a new chain.


nit: "The initial configuration..."

Perhaps InitialChainConfig is a better name as a result?

ma2bd · 2025-04-16T20:06:28Z

linera-base/src/identifiers.rs


+    /* TODO: figure out if this can be fixed.


AFAICT you can remove

(We could take one example chain description and assert its hash, just so we can't inadvertently change how chain IDs are computed. Not sure if that's useful.)

afck · 2025-04-17T09:02:24Z

linera-core/src/client/mod.rs

+                    RemoteNode::download_blobs(&blob_ids, validators, self.blob_download_timeout)
+                        .await
+                        .ok_or(LocalNodeError::BlobsNotFound(blob_ids))?;
+                self.local_node.storage_client().write_blobs(&blobs).await?;


But now the remote node could have tricked us, and the chain might not actually have been created.

I think we need to use something like ChainClient::update_local_node_with_blobs_from, which downloads proof from the validators that the blob was created. Except if we already have at least one correctly signed certificate from this chain—its first block can also count as a proof, I guess (if we double-check the committee information with the admin chain).

Hmm, good point 🤔

afck · 2025-04-17T09:05:57Z

linera-core/src/client/mod.rs

+        let mut validators_vec = validators.iter().collect::<Vec<_>>();
+        validators_vec.shuffle(&mut rand::thread_rng());
+        for remote_node in validators_vec {
+            let info = self.chain_info(chain_id, validators).await?;


Actually, at this point we only need the next block height. Why not just set it to zero if we don't have the chain at all?

afck · 2025-04-17T09:11:58Z

linera-core/src/client/mod.rs

+        let blob = RemoteNode::download_blob(&nodes, chain_desc_id, self.blob_download_timeout)
+            .await
+            .ok_or(LocalNodeError::BlobsNotFound(vec![chain_desc_id]))?;
+        self.local_node.storage_client().write_blob(&blob).await?;


Same as above: We can't trust that blob yet.

afck · 2025-04-17T09:14:21Z

linera-core/src/client/mod.rs

-            // our inbox, and include it before all other messages.
-            rearranged = IncomingBundle::put_openchain_at_front(&mut pending_message_bundles);
-            ensure!(rearranged, LocalNodeError::InactiveChain(self.chain_id));
-        }


(Nice! Much less messy now! 🎉)

afck · 2025-04-17T09:18:41Z

linera-core/src/remote_node.rs

@@ -314,7 +314,7 @@ impl<N: ValidatorNode> RemoteNode<N> {
    }

    #[instrument(level = "trace", skip(validators))]
-    async fn download_blob(
+    pub async fn download_blob(


Let's add a doc comment that this downloads a blob but doesn't verify that it has actually been published and accepted by a quorum of validators.

afck · 2025-04-17T09:35:38Z

linera-chain/src/chain.rs

+            ChainError::InvalidBlockTimestamp
+        );
+
+        self.ensure_is_active(local_time).await?;


I think we should do that before ensuring the ensure! above: If the chain is initialized with a timestamp, we should ensure that the first block's timestamp is no earlier than that.

So maybe the ensure! and set should go back into execute_block_inner?

afck · 2025-04-17T09:38:30Z

linera-chain/src/block.rs

@@ -508,6 +513,8 @@ impl Block {
        self.oracle_blob_ids().contains(blob_id)
            || self.published_blob_ids().contains(blob_id)
            || self.created_blob_ids().contains(blob_id)
+            || (blob_id.blob_type == BlobType::ChainDescription
+                && blob_id.hash == self.header.chain_id.0)


We could do that only if self.header.height is zero. It's sufficient if the first block requires the blob, and it might create less overhead managing blobs later. Also, then it matches required_blob_ids again.

afck · 2025-04-17T09:40:59Z

linera-base/src/identifiers.rs


+    /* TODO: figure out if this can be fixed.


(We could take one example chain description and assert its hash, just so we can't inadvertently change how chain IDs are computed. Not sure if that's useful.)

afck · 2025-04-17T09:46:27Z

linera-base/src/data_types.rs

+    /// The epoch in which the chain is created.
+    pub epoch: Epoch,
+    /// Blob IDs of the committees corresponding to epochs.
+    pub committees: BTreeMap<Epoch, Vec<u8>>,


Then this blob depends on other blobs, which complicates chain initialization, so we thought we'd put the full committees in here for now. We could change that as an optimization later.

afck · 2025-04-17T09:55:07Z

linera-base/src/data_types.rs

+pub struct ChainDescription {
+    origin: ChainOrigin,
+    timestamp: Timestamp,
+    config: OpenChainConfig,


We should consider inlining this type here. (Probably in a later PR.)

ma2bd · 2025-04-16T20:09:34Z

linera-chain/src/chain.rs

@@ -445,11 +443,30 @@ where
    }

    /// Invariant for the states of active chains.
-    pub fn ensure_is_active(&self) -> Result<(), ChainError> {
+    pub async fn ensure_is_active(&mut self, local_time: Timestamp) -> Result<(), ChainError> {


nit: maybe due for a rename

ma2bd · 2025-04-16T20:11:32Z

linera-chain/src/data_types.rs

-
-    /// Rearranges the messages in the bundle so that the first message is an `OpenChain` message.
-    /// Returns whether the `OpenChain` message was found at all.
-    pub fn put_openchain_at_front(bundles: &mut [IncomingBundle]) -> bool {


ma2bd · 2025-04-18T14:44:56Z

linera-base/src/data_types.rs

+    /// The epoch in which the chain is created.
+    pub epoch: Epoch,
+    /// Blob IDs of the committees corresponding to epochs.
+    pub committees: BTreeMap<Epoch, Vec<u8>>,


The comment is inaccurate then, right?

incoming bundle

ma2bd reviewed Apr 16, 2025

View reviewed changes

afck reviewed Apr 17, 2025

View reviewed changes

bart-linera force-pushed the issue-2390 branch from ab76ddd to f59a50e Compare April 18, 2025 14:21

ma2bd reviewed Apr 18, 2025

View reviewed changes

Change ChainDescriptions into blobs

06bde66

bart-linera force-pushed the issue-2390 branch from f59a50e to 06bde66 Compare April 23, 2025 11:09

bart-linera added 2 commits April 23, 2025 15:03

Improve some error messages

b219394

Ensure the sender's ChainDescription is present when processing an

71af9ed

incoming bundle

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Change ChainDescriptions into blobs #3787

Change ChainDescriptions into blobs #3787

bart-linera commented Apr 16, 2025

ma2bd Apr 16, 2025

afck Apr 17, 2025

This comment was marked as spam.

ma2bd Apr 18, 2025

afck Apr 18, 2025

This comment was marked as spam.

ma2bd Apr 16, 2025

ma2bd Apr 16, 2025

afck Apr 17, 2025

afck Apr 17, 2025

bart-linera Apr 17, 2025

afck Apr 17, 2025

afck Apr 17, 2025

afck Apr 17, 2025

afck Apr 17, 2025

afck Apr 17, 2025

afck Apr 17, 2025

afck Apr 17, 2025

afck Apr 17, 2025

afck Apr 17, 2025

ma2bd Apr 16, 2025

ma2bd Apr 16, 2025

ma2bd Apr 18, 2025

Change ChainDescriptions into blobs #3787

Are you sure you want to change the base?

Change ChainDescriptions into blobs #3787

Conversation

bart-linera commented Apr 16, 2025

Motivation

Proposal

Test Plan

Release Plan

Links

Choose a reason for hiding this comment

Choose a reason for hiding this comment

This comment was marked as spam.

Choose a reason for hiding this comment

Choose a reason for hiding this comment

This comment was marked as spam.

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment